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I.  Problem 


For  the  past  decade,  behavioral  objectives  have  been  widely  used 
by  instructional  designers,  I'esearchers,  evaluators,  and  curriculum 
designers  for  systematic  planning  and  assessment  in  education.  Most 
advocates  of  behavioral  objectives  agree  that  all  adequate  behavioral 
objectives  must  contain  three  basic  components:  (1)  the  verb,  which  states 

what  the  learner  will  be  able  to  do  after  the  learning  experience  (e.g., 
to  measure);  (2)  the  condition  of  performance,  which  states  the  circum- 
stances under  which  the  action  occurs  (e.g.,  given  a ruler);  and  (3)  the 
criterion,  which  states  the  standard  of  performance  (e.g.,  to  the  nearest 
quarter  of  an  inch).  Although  there  is  general  agreement  at  the  conceptual 
level  as  to  what  constitutes  an  adequate  behavioral  objective,  little  has 
been  done  in  the  way  of  empirical  research  to  develop  an  operational 
definition  of  the  concept.  Furthermore,  there  has  been  little  research  to 
determine  the  variables  that  govern  the  efficacy  of  behavioral  objectives. 
One  result  of  this  situation  seems  to  be  that  many  individuals  act  as  if 
any  explicit  statement  of  outcomes  will  serve  as  an  adequate  behavioral 
objective.  Consequently,  behavioral  objectives  exhibit  a wide  variety  of 
characteristics,  characteristics  which  on  occasion  bring  into  question  the 
appropriateness  of  the  label  "behavioral." 

In  the  past,  both  research  and  development  efforts  on  objectives 
have  focused  on  the  role  of  the  verb,  to  the  exclusion  of  the  other  two 
characteristics.  This  emphasis  seems  to  have  been  based  on  the  assumption 
that  the  verb  is  the  primary,  if  not  the  sole,  determiner  of  the  degree  to 
which  an  objective  may  be  considered  behavioral,  since  it  is  the  verb  that 
represents  what  the  person  "does."  In  one  important  study,  Deno  and 
Jenkins  (1969)  had  14  in-service  educators  rate  99  verbs  on  a five-point 
rating  scale  of  observability.  These  99  verbs  were  selected  from  a 


2 

well-known  experimental  curriculum.  The  results  indicated  that  many  widely 
used  and  highly  recommended  behavioral  terms  refer  to  behavior  not  per- 
ceived by  educators  to  be  clearly  observable.  Deno  and  Jenkins  concluded 
that  verbs  used  in  behavioral  objectives  are  selected  for  consistency  of 
usage  rather  than  for  maximum  observability.  This  and  the  scattering  of 
other  empirical  studies  on  the  role  of  the  verb  in  behavioral  objectives 
have  been  summarized  by  MacDonal d-Ross  (1973). 

The  study  reported  here  is  one  of  a continuing  series  of  studies 
analyzing  raters'  perceptions  of  the  components  of  behavioral  objectives, 
rated  both  in  isolation  and  within  complete  statements  of  objectives. 

Since  these  studies  have  enabled  the  investigators  to  assess  the  degree  to 
which  various  components  contribute  to  the  raters'  perceptions  of  the 

cr  'bjective,  they  provide  a step  toward  an  operational  definition  of 

dvioral  objective. 

In  the  first  study  of  this  series,  Gerlach  (1974)  replicated  the 
Deno  and  Jenkins  study  by  having  35  seniors  and  first-year  graduate  students 
in  an  Education  course  at  the  University  of  Minnesota  rate  the  same  99 
verbs  on  a scale  from  most  observable  (1)  to  least  observable  (5).  Results 
were  essentially  the  same  as  those  obtained  by  Deno  and  Jenkins,  with  a 
correlation  between  the  two  sets  of  ratings  of  +.90. 

Both  the  Deno  and  Jenkins  and  the  Gerlach  studies  assessed  the 
influence  of  only  one  component,  the  verb,  rated  in  isolation.  The  next 
logical  step  was  to  explore  whether  or  not  statements  of  conditions  and 
of  criteria  would  influence  individuals'  perceptions  of  the  observability 
of  behavioral  objectives  based  on  a given  verb.  For  example,  if  a verb  that 
has  a low  observability  rating  when  rated  in  isolation  is  used  in  an 
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objective,  will  that  objective  receive  a higher  rating  if  the  objective 
includes  highly  precise,  observable  statements  of  condition  and/or  criterion? 

Gerlach  and  Barron  (1974)  pursued  this  line  of  inquiry  in  a study 
designed  to  assess  the  roles  of  the  three  basic  components  of  an  objective, 
rated  in  isolation  as  well  as  in  complete  statements  containing  all  the 
components.  The  results  confirmed  the  importance  of  the  verb  in  the  per- 
ception of  objectives,  but  they  also  provided  evidence  that  the  choice  of 
condition  and  criterion  influences  the  rating  of  the  complete  objective. 

In  an  extension  of  this  study,  Barron  and  Gerlach  (1975)  used  four  different 
rating  scales  (observable-nonobservable,  precise-vague,  clear-unclear,  and 
concrete-abstract)  to  examine  the  differences  among  rating  dimensions.  The 
results  confirmed  the  importance  of  conditions  and  criteria  in  determining 
the  overall  objective  rating.  In  addition,  the  four  rating  scales  were 
highly  intercorrel ated,  suggesting  that  they  can  be  used  almost  inter- 
changeably. 

In  a multiple  correlation  analysis  of  the  Barron  and  Gerlach  (1975) 
data,  it  was  found  that  a substantial  portion  of  the  variance  of  ratings 
of  total  objectives  could  be  accounted  for  by  a linear  combination  of  the 
three  component  parts,  verb,  condition,  and  criterion.  The  variance 
unaccounted  for  was  a nagging  concern,  however,  and  led  to  a reanalysis  of 
the  problem.  In  the  reanalysis,  it  became  clear  that  the  role  of  the 
direct  object  was  being  ignored.  Up  to  this  point,  all  studies  had  used 
"x"  and  "y"  inserted  where  the  direct  object  would  normally  go,  e.g.,  "to 
draw  X accurately  from  memory."  This  was  done  to  avoid  interference  from 
choice  of  specific  subject  matter;  in  many  practical  cases,  the  direct 
object  constitutes  a "given"  that  is  ordinarily  not  subject  to  variation 
by  the  instructional  designer.  To  the  degree  that  subjects  mentally  inserted 
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their  own  choice  of  direct  object,  the  overall  rating  may  have  been  affected. 
For  example,  a subject  who  thinks  in  terms  of  fairly  concrete  objects  such 
as  "a  square"  might  rate  the  overall  objective  as  much  more  precise  and 
observable  than  one  who  mentally  inserts  the  word  "something."  Furthermore, 
it  seemed  possible  that  the  use  of  the  unspecified,  abstract  letters  as 
direct  objects  might  have  caused  a general  shift  toward  perception  of 
obje'-tives  as  less  observable  or  less  precise.  For  these  reasons,  it  seemed 
desirable  to  determine  the  strength  of  association  between  the  choice  of 
direct  object  and  the  overall  rating  of  the  complete  objective. 

The  present  study  was  designed  to  explore  the  role  of  the  direct 
object  in  the  perception  of  the  complete  behavioral  objective.  The  study 
was  designed  to  answer  the  following  specific  questions; 

(1)  Does  the  inclusion  of  direct  objects,  as  opposed  to  abstract 
direct  objects  (i.e.,  "x"  or  "y"),  in  statements  of 
behavioral  objectives  generate  an  overall  shift  of  ratings 
in  the  direction  of  increased  observability  and  precision? 

(2)  Do  the  ratings  of  individual  objectives  change  as  a result 
of  the  choice  of  different  direct  objects? 

Since  it  could  be  argued  that  a positive  answer  to  the  second  ques- 
tion would  cloud  the  interpretation  of  a positive  answer  to  the  first,  the 
lists  used  were  balanced  by  selecting  direct  objects  that  covered  a wide 
range  of  observability.  In  this  way,  it  was  hoped  to  avoid  the  objection 
that  any  overall  shift  was  caused  by  using  highly  observable  direct  objects 
throughout. 

II.  Method 

Subjects  and  Design 

Sixty-four  undergraduate  students  from  two  upper  division  instruc- 
tional media  courses  participated  in  this  study  as  part  of  their  regular 


class  sessions.  Four  additional  students  omitted  items  from  their  response 
booklets,  rendering  these  booklets  useless;  their  data  were  excluded  from 
the  study. 
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Forty-four  of  these  subjects  served  in  the  main  design  of  the  study. 
Each  of  these  subjects  rated  24  objective  statements  containing  24  different 
direct  objects.  Each  subject  also  rated  the  24  direct  objects  in  isolation 
for  purposes  of  comparison.  The  24  objects  were  organized  into  six  objec- 
tive sets  of  four  objectives  each.  Within  each  set,  the  verb,  condition, 
and  criterion  were  identical,  so  that  the  four  objectives  varied  only  in 
choice  of  direct  object.  Different  sets  used  different  verbs  conditions, 
and  criteria,  and  no  component  of  an  objective  was  used  in  twa  different 
objective  sets.  The  arrangement  of  components  is  shown  in  Table  , from 
which  the  entire  set  of  24  objectives  can  be  reconstructed.  The  experi- 
mental design  was  a repeated  measures  nested  design,  with  six  objective  sets 
and  (four  different)  direct  objects  nested  within  objective  sets.  This 
design  was  replicated  with  two  rating  dimensions,  observability  and  precision; 
22  subjects  were  used  for  each  rating  dimension.  The  ratings  of  direct 
objects  in  isolation  were  collected  for  the  purpose  of  determining  the  corre- 
lation between  direct  objects  and  complete  objectives  containing  those  direct 
objects. 

In  addition  to  the  main  design,  20  additional  subjects  rated  six 
complete  objectives  containing  "x"  or  "y"  in  place  of  the  direct  object. 

These  six  objectives  corresponded  to  the  six  objective  sets  used  in  the 
main  design.  These  ratings  were  collected  to  compare  overall  ratings  with 
and  without  explicit  direct  objects.  Again,  half  the  subjects  rated 
observability  and  half  precision. 


Components  of  the  Six  Objective  Sets 


To  understand  given  audible  without  error  computer  processes... 

encouragement  nature  of  a scientific. 

costs  of  education... 
idiomatic  expressions.. 


Materials  and  Procedure 


Subjects  in  the  main  design  each  received  a booklet  containing  four 
pages  of  complete  objectives,  six  to  a page.  On  each  page,  one  objective 
from  each  objective  set  appeared.  A fifth  page  contained  the  24  direct 
objects  to  be  rated  in  isolation.  The  rating  scale  to  be  used  was  printed 
at  the  top  of  each  page  in  the  booklet,  in  a diagram  depicting  the  full 
range  of  the  scale,  most  observable  (1)  to  least  observable  (5)  or  precise 
(1)  to  vague  (5).  The  same  rating  dimension  was  used  throughout  each  booklet. 
Subjects  recorded  their  responses  directly  in  the  booklets.  The  booklets 
were  assembled  in  counterbalanced  order,  to  avoid  order  effects,  and  were 
stapled  to  a cover  page  which  provided  instructions  and  two  sample  ratings.* 

For  the  20  subjects  rating  objectives  without  direct  objects,  the 
six  objectives  to  be  rated  were  all  on  a single  page;  the  same  cover 
sheet  was  used  for  these  booklets. 

The  24  direct  objects  used  in  this  study  were  selected  from  objec- 
tives developed  by  the  National  Assessment  Project  (1972).  The  pool  of 
direct  objects  was  rated  a priori  by  the  experimenters  for  observability 
and  precision;  the  final  24  selected  provided  a wide  range  of  values  within 
each  objective  set.  The  six  verbs  used  (cf.  Table  1)  were  selected  from 
a pool  of  verbs  that  are  not  domain  specific,  and  for  which  earlier  studies 
had  provided  data  concerning  the  perceived  observability,  both  in  isolation 
and  in  context.  They  were  chosen  so  that  the  mean  ratings  covered  the 


*A11  subjects  also  rated  25  verbs,  25  conditions,  and  25  criteria 
as  part  of  a replication  of  previous  work.  The  results  from  these  ratings 
generally  confirm  previous  results,  and  they  will  not  be  discussed  further 
in  this  report. 
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entire  spectrum  of  the  observability  scale.  The  conditions  and  criteria 
were  chosen  in  similar  fashion.  The  components  were  combined  into  objectives 
in  such  a way  as  to  avoid  absurdities,  and  to  be  consistent  with  generally 
accepted  grammatical  and  contextual  conventions. 

The  subjects  were  given  oral  and  written  instructions  by  the 
experimenter  at  the  outset,  along  with  the  examples  explaining  the  method 
of  rating.  Specific  reference  was  made  in  the  instructions  to  the  fact  that 
these  stdteiients  are  typical  of  expressions  used  in  instructional  objectives. 
The  subjects  were  also  instructed  not  to  refer  back  to  a page  once  it  had 
been  completed. 


III.  Results 


The  means  and  standard  deviations  for  objective  sets,  with  and 
without  direct  objects,  are  shown  in  Table  2 separately  for  each  rating 
dimension.  It  is  readily  apparent  that  there  is  no  significant  difference 
between  subjects'  ratings  of  objectives  with  direct  objects  and  objectives 
containing  "x"  and  "y".  This  is  reflected  in  tests  computed  separately  for 
observability,  1(30)  = -60,  and  precision,  t^30)  = 1.18. 

Turning  to  the  question  of  differences  induced  by  choice  of  direct 
object,  an  analysis  of  variance  was  computed  separately  for  each  of  the 
two  rating  dimensions;  this  was  done  for  purposes  of  both  simplicity  and 
clarity.  The  results  of  the  two  analyses  are  shown  in  Tables  3 and  4.  As 
expected,  since  an  effort  had  been  made  to  construct  objective  sets  differing 
in  ratings,  the  mean  ratings  for  verb  sets  differed  significantly;  in  the 
observability  analysis,  the  most  observable  objective  was  "to  accurately 
draw  a graph  from  memory"  and  the  least  observable  objective  was  "to  under- 


Means  and  Standard  Deviations  for  Objective  Sets 


Observabil ity  Precision 


X 

S^ 

N 

X 

S 

N 

Objectives  containing 
X and  y in  place  of 

direct  objects 
Objectives  with  direct 

3.012 

.4754 

10 

3.046 

.6164 

10 

objects 

2.890 

.5385 

22 

2.760 

.6132 

22 

Analysis  of  Variance  Summary  Table  for  Observability 


Source 

§1 

df 

MS 

F 

Subjects 

145.9773 

21 

6.951 

Verb  Sets 

119.197 

5 

23.839 

10.502* 

Direct  Objects  within 
Verb  Sets 

40.8636 

18 

2.270 

2.4059* 

Subjects  by  Verb  Sets 

303.3863 

105 

2.8894 

Residual 

356.6364 

378 

.9435 

TOTAL 


966.0606 


Analysis  of  Variance  Sutmiary  Table  for  Precision 


Source 

§1 

F 

Subjects 

189.3106 

21 

9.0148 

Verb  Sets 

47.3333 

5 

9.4666 

5.28* 

Direct  Objects  within 
Verb  Sets 

32.2955 

18 

1.7942 

2.27* 

Subjects  by  Verb  Sets 

261.6667 

105 

2.4921 

Residual 

299.2045 

378 

.7915 

TOTAL 


829.8106 
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precision  analysis,  the  most  precise  objective  was  "to  draw  simple  floor 
plans  of  furnishings  in  rooms,  from  memory"  and  the  least  precise  was  "to 
appropriately  identify  stated  and  unstated  assumptions,  given  various 
chances."  More  important  was  the  question  of  whether  the  direct  objects 
caused  significant  differences  within  objective  sets,  that  is,  whether  the 
four  objectives  within  objective  sets  differed.  The  results  are  clear  for 
both  rating  dimensions.  The  objectives  within  sets  differ  significantly  more 
than  would  be  expected  by  chance;  for  observability,  F^-|g  = 2.41.  -01; 

for  precision,  F^ig  = 2.27,  E finding  is  further  bolstered 

by  the  correlations  between  ratings  of  direct  objects  and  ratings  of  objec- 
tives containing  those  direct  objects,  computed  within  sets  and  averaged: 

+.78  for  observability  ratings  and  +.41  for  precision  (^.'s  < .05). 

IV.  Discussion 

The  results  indicate  clearly  that  the  choice  of  direct  object  does 
influence  raters'  perceptions  of  the  observability  and  precision  of  a 
behavioral  objective.  Taken  together  with  the  results  of  Barron  and  Gerlach 
(1975),  these  results  provide  convincing  evidence  that  no  single  component, 
such  as  the  verb,  should  be  singled  out  as  being  of  primary  importance  in 
determining  the  character  of  a behavioral  objective.  Rather,  careful  atten- 
tion must  be  paid  to  all  components  to  insure  an  objective  that  is  observable, 
precise,  clear,  and  concrete.  This  conclusion  must  be  tempered  by  the 
realization  that  the  instructional  designer  does  not  have  complete  freedom 
in  selecting  direct  objects.  Instead,  they  are  often  specified  by  the 
user  organization  as  part  of  the  instructional  goal.  For  example,  if  the 
aim  is  to  teach  the  student  to  multiply  fractions,  there  is  little  opportunity 
to  substitute  another  direct  object.  However,  once  alerted  to  the  problem, 
the  instructional  designer  will  have  no  difficulty,  having  recognized  the 
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inherent  imprecision  of  asking  the  child  to  "draw  a nice  picture,"  in  seeking 
a more  precise,  observable  direct  object. 

The  absence  of  a pronounced  shift  toward  greater  observability  and 
precision  for  objectives  containing  direct  objects  suggests  that  subjects 
either  (1)  substitute  mentally  their  own  direct  objects,  which  average  out 
to  about  the  same  values  as  real  direct  objects,  and/or  (2)  ignore  the 
"x"  as  an  active  element  in  the  objective  when  they  have  not  been  exposed 
to  other  sentences  containing  real  direct  objects.  Regardless  of  the  inter- 
pretation, the  lack  of  a radical  shift  lends  increased  confidence  in  results 
obtained  in  previous  studies  employing  only  "x"  and  "y"  in  objectives. 

The  empirical  data  gathered  in  this  study  on  the  ratings  of  the 
components  of  a behavioral  objective  in  isolation  and  within  complete  state- 
ments of  objectives  indicate  that  the  selection  of  the  various  components 
influences  individuals'  perceptions  of  objectives.  The  data  suggest  that 
the  investigators  are  moving  closer  towards  a consistent  operational  defini- 
tion of  the  behavioral  objective.  As  the  function  of  the  individual  compo- 
nents becomes  more  clearly  delineated,  the  future  research  in  this  area  will 
explore  how  this  type  of  information  can  assist  educators  and  trainers  to 
select  and  to  construct  more  precise  statements  of  behavioral  objectives. 
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